Bridging AI and Economics

Evaluating the Feasibility of RAG Models for Regulatory Compliance

Jesús Martínez del Rincón
Abhishek Pramanick
Barry Quinn

2024-11-20

Introduction

Collaborative Project Overview
- Led by Queen’s University Belfast researchers.
- Supported by UKRI through the UKFin+ programme.
- Aim: Evaluate the feasibility of AI (RAG models) for compliance in investment management.

Objectives

  1. Explore AI applications for compliance efficiency.
  2. Conduct economic evaluations of cost and resource use.
  3. Focus on reporting, risk, and monitoring improvements.
  4. Address challenges like adaptability and integration.

Methodology Overview

  1. Cost-Benefit Analysis: Evaluate direct and indirect cost impacts.
  2. Process Mapping: Assess workflow efficiencies.
  3. Simulation Modelling: Test adaptability to evolving regulations.
  4. Stakeholder Interviews: Capture practical insights from compliance professionals.

Preliminary Insights

  • Efficiency Gains: Improved task completion times.
  • Cost Reduction: Lower resource requirements for key compliance tasks.
  • Stakeholder Feedback: Highlighted potential for better interpretability and usability.

Work Packages

  • WP1: Define use cases and baseline metrics.
  • WP2: Train RAG models for Q/A tasks.
  • WP3: Automate rule extraction via OWL ontology.
  • WP4: Address inconsistencies in rule sets.
  • WP5: Collect qualitative insights and finalise the CBA.

Challenges and Limitations

  • Data Availability: Ensuring comprehensive regulatory data.
  • Model Trust: Balancing accuracy with explainability.
  • Scalability: Adapting to evolving compliance requirements.

Ethical Considerations

  • Fairness: Address potential biases in AI outputs.
  • Transparency: Use explainable AI (XAI) methods for clarity.
  • Privacy: Comply with GDPR and other data protection laws.
  • Human Oversight: Support, not replace, compliance professionals.

Future Work

  1. Develop specialised financial language models.
  2. Refine retrieval mechanisms to improve accuracy.
  3. Study long-term economic impacts of RAG models.
  4. Create ethical guidelines for AI in compliance.

Thank You!

Contact Information
Dr Barry Quinn
Queen’s University Belfast
📧 barry.quinn@qub.ac.uk